Asynchronous Parallel Schemes: A Survey
نویسندگان
چکیده
It is well known that synchronization and communication delays are the major sources of performance degradation of synchronous parallel algorithms. It has been shown that asynchronous implementations have the potential to reduce the overhead to minimum. This paper surveys the existing asynchronous schemes and the suucient conditions for the convergence of the surveyed schemes. Some comparisons among these schemes are also presented.
منابع مشابه
Parallel Iterative Discontinuous Galerkin Finite-element Methods
We compare an iterative asynchronous parallel algorithm for the solution of partial diierential equations, with a synchronous algorithm , in terms of termination detection schemes and performance. Both algorithms are based on discontinuous Galerkin nite-element methods, in which the local elements provide a natural decomposition of the problem into computationally-independent sets. We demonstra...
متن کاملParallel and Distributed Simulation of Discrete Event Systems
The achievements attained in accelerating the simulation of the dynamics of complex discrete event systems using parallel or distributed multiprocessing environments are comprehensively presented. While parallel discrete event simulation (DES) governs the evolution of the system over simulated time in an iterative SIMD way, distributed DES tries to spatially decompose the event structure underl...
متن کاملPerformance analysis of different checkpointing and recovery schemes using stochastic model
Several schemes for checkpointing and rollback recovery have been reported in the literature. In this paper, we analyze some of these schemes under a stochastic model. We have derived expressions for average cost of checkpointing, rollback recovery, message logging and piggybacking with application messages in synchronous as well as asynchronous checkpointing. For quasi-synchronous checkpointin...
متن کاملParallel Shortest Path Auction Algorithmsl
In this paper we discuss the parallel implementation of the auction algorithm for shortest path problems. We show that both the one-sided and the two-sided versions of the algorithm admit asynchronous implementations. We implemented the parallel schemes for the algorithm on a shared memory machine and tested its efficiency under various degrees of synchronization and for different types of prob...
متن کاملDemystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis
DeepNeural Networks (DNNs) are becoming an important tool inmodern computing applications. Accelerating their training is a major challenge and techniques range from distributed algorithms to low-level circuit design. In this survey, we describe the problem from a theoretical perspective, followed by approaches for its parallelization. Specifically, we present trends in DNN architectures and th...
متن کامل